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1.  Introduction 


The  17th  annual  conference  on  Multi- Criteria  Decision  Making  was  held  at  the  Whistler 
Conference  Centre  from  August  6  to  11,  2004.  There  were  around  150  participants  and  as 
many  presentations,  spread  across  4  parallel  sessions.  Participants  came  from  many  different 
fields,  most  notably  from  economics,  management  science,  business  administration, 
operations  research,  mathematics,  software  engineering  and  industrial  engineering.  This 
made  for  very  different  presentations,  some  concerning  new  research  ideas,  others 
applications  and  case  studies,  theoretical  foundations,  tutorials  or  software  surveys. 


2.  Overview  of  the  Areas  of  Research  and  Application 

Although  there  were  many  different  topics,  most  presentations  were  related  to  one  of  the 
following  areas: 

1 .  Group  Decision  Support  Methods; 

2.  Decision  Support  Methods; 

3.  AHP  applications; 

4.  Multi-Objective  Linear  Programming  (MOLP)  algorithms; 

5.  Industrial  engineering  applications; 

6.  Behavioural  considerations,  and 

7.  Fuzzy  MCDM. 


3.  Topics  for  Further  Research 

Of  the  many  subjects  presented,  the  following  appeared  to  be  of  potential  interest  to  ORD. 
Further  research  into  some  of  those  areas  could  provide  insight  in  the  way  we  deal  with  multi¬ 
criteria  decision  problems.  Further  details  concerning  these  areas  and  their  application  to 
specific  ORD  projects  will  be  discussed  in  phase  2  of  the  Maritime  OR  Toolset  Review. 
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1.  Dempster-Shafer  theory 


The  theory  developed  by  Dempster  and  later  by  Shafer  is  an  extension  of 
Bayesian  theory.  It  defines  a  set  of  rules  and  operators  for  combining 
evidence  under  uncertainty.  Dempster-Shafer  theory  was  also  a  recurring 
theme  at  the  INFORMS/CORS  conference,  especially  from  professors  at  the 
US  Military  Academy  in  West  Point,  NY.  Among  other  things,  they  use  this 
logical  paradigm  to  model  sensor  information  flow,  in  particular  in  a  network 
centric  context.  For  MCDM,  it  is  at  the  core  of  the  Evidential  Reasoning 
approach  and  the  focus  of  some  research  on  multi-criteria  group  decision 
making  to  combine  evidence  from  decision  makers  and  evidence  from 
criteria. 

2.  Behavioural  Considerations  and  Impact  of  Cognitive  Biases  on  MCDM 

Results  presented  by  several  speakers  indicated  that  behavioural 
considerations  play  an  important  role  in  decision  making.  This  is  especially 
important  when  using  software  like  AHP  or  when  constructing  questionnaires 
for  SME’s  (  see  [10]  for  many  examples). 

One  particular  study  [  1  ]  showed  that  the  number  of  mechanical  errors 
produced  by  decision  makers  increased  greatly  with  the  size  of  the  problem 
when  using  AHP,  sometimes  enough  to  counter  the  positive  effect  of  using  a 
Decision  Support  System.  Mechanical  errors  are  typos  and  other  errors  that 
result  from  lapses  in  the  user’s  attention. 

Another  study  [2]  suggested  that  of  all  types  of  cognitive  biases,  anchoring  is 
one  that  can  severely  skew  the  results  of  the  decision  analysis.  For  example, 
the  two  questions  “How  many  CP- 140  patrol  area  A  weekly?”  and  “How 
many  CP- 140  do  you  think  are  needed  weekly  to  patrol  area  A?”  should  not 
be  asked  in  this  sequence  in  a  questionnaire  or  interview.  The  answer  to  the 
second  question  could  be  dependent  of,  i.e.  anchored  to,  the  answer  to  the 
first  question.  Research  shows  that  the  anchoring  happens  frequently  and  has 
a  strong  impact  on  decision  making. 

Another  particularly  interesting  presentation  [3]  insisted  that  good  decisions 
are  made  with  intuition  and  emotion.  The  author  referred  to  neuro-biological 
studies  to  enforce  this  point.  The  main  criticism  was  aimed  at  methods  like 
AHP  that  use  pairwise  comparisons  to  “simplify  the  cognitive  burden”  on  the 
user.  The  speaker  argues  that  this  creates  artificial  scenarios  that  prevent  the 
user  from  making  intuitive  decisions.  The  conclusion  of  the  speaker, 
expressed  in  his  own  terms,  is  the  following:  “  The  challenge,  therefore,  is  to 
develop  processes  that  are  able  to  still  present  such  trade-off  problems  in  a 
way  that  is  vivid  enough  to  elicit  emotions  when  the  decision-maker  ponders 
the  alternatives.” 
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3.  Integration  of  Discrete  Event  Simulation,  Evolutionary  Multi-Objective 
Optimisation  and  Multi-Criteria  Decision  Making 

Currently  still  at  the  early  stages  of  development,  this  project  seeks  to  use 
MCDM  to  facilitate  the  exploration  of  the  parameter  space  in  Discrete  Event 
Simulations  (DES).  Typically,  DES  (like  MARVIN  or  any  ARENA  model) 
is  used  in  a  trial  and  error  fashion  to  explore  the  space  of  possible  outcomes. 
This  works  well  when  the  system  under  study  is  fairly  small  and  if  there  is 
only  one  criterion  to  measure  performance,  the  problem  becomes  much  more 
complex  when  performance  is  measured  by  conflicting  criteria.  Research  in 
this  area  is  looking  at  combining  DES  with  multi- objective  evolutionary 
optimization  (like  Genetic  Algorithms)  to  reduce  the  space  of  parameters  to 
explore.  A  MCDM  system  could  provide  the  choice  for  the  preferred 
solution  from  this  reduced  set  of  outcomes  and  provide  a  strategy  for 
improvement  (a  new  search  direction).  This  decision  support  system  would 
be  interactive. 

Some  of  these  ideas  have  been  applied  to  chemical  process  design  using  a 
commercial  numerical  simulator  and  an  interactive  multi-objective 
optimization  method  called  NIMBUS  (see  [4]). 

4.  Cooperative  Games  for  Auto-Scheduling 

Although  this  area  is  in  no  way  a  trend  (there  was  only  one  presentation  on 
this  subject  ([5])  ),  it  is  a  different  and  unusual  application  of  decision 
support.  The  problem  presented  consisted  in  finding  the  optimal  schedule  for 
a  set  of  machines  in  a  shop  with  resource  and  time  constraints.  The  solution 
proposed  was  inspired  from  game  theory  and  modeled  each  machine  as  a 
player  cooperating  with  the  other  players  (machines)  to  achieve  its  own  goal, 
i.e.  minimize  the  penalties  associated  with  the  infringement  of  the 
constraints.  Each  player-machine  was  associated  to  a  software  agent  and  a 
super-agent  was  tasked  with  the  negotiation  process  between  the  players,  in 
other  words  the  group  decision  support.  This  procedure  is  fully  automated 
and  implemented  in  Java,  it  produces  schedules  in  a  few  seconds  for 
relatively  small  problems. 


4.  Software  and  Methods 


A  number  of  software  tools  and  methods  were  mentioned  during  the  conference.  They  seem 
to  correspond  to  different  ‘schools  of  thought’  rather  than  to  apply  to  specific  problems.  This 
is  a  short  list  of  the  most  frequently  mentioned.  It  is  of  interest  to  note  that  a  survey  of 
commercial  tools  for  MCDM  appears  annually  in  the  publication  “OR/MS  Today”. 

1.  Interactive  Decision  Support  (IDS) 
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Uses  the  Evidential  Reasoning  approach,  based  on  Dempster- Shafer  theory. 
Intervals  and  uncertainty  are  handled  in  the  decision  matrix.  [13]; 

2.  Multi-Attribute  Value  Theory  (MAVT):  HiView,  VISA 

Very  interactive  tools  that  allow  the  user  to  visualize  the  input  data  and 
manipulate  it  with  the  mouse,  also  provides  some  sensitivity  analysis; 

3.  Analytical  Hierarchy  Process  (AHP):  ExpertChoice 

Based  on  Saaty’s  method  of  pairwise  comparisons  to  elicit  weights  for 
criteria  and  performance  scores  of  options  on  the  different  criteria; 

4.  Hybrid  MAVT  &  AHP:  WebHipre,  Logical  Decisions 

Those  methods  are  similar  in  format  to  the  methods  of  the  second  category 
but  they  also  use  pairwise  comparisons.  WebHipre  is  meant  to  be  used  over 
the  internet,  and  is  based  on  the  Hipre3+  method; 

5.  knowCube 

Different  type  of  software  that  uses  a  ‘spider’  graph  to  explore  the  space  of 
options  and  attributes.  Highly  interactive.  [14];  and  , 

6.  Outranking  methods:  PROMETHEE  and  ELECTRE 

From  the  ‘French’  school  of  MCDM.  Those  methods  essentially  consist  in 
asking  the  decision  maker  to  rank  the  options  relative  to  each  criterion,  the 
method  then  tries  to  outrank  the  options  to  find  a  dominating  one.  These  are 
the  methods  that  resemble  the  most  the  tau-x  based  method  used  at  ORD 
[1.15]. 


5.  Overall  Impression 

Overall,  this  conference  was  a  great  opportunity  to  get  an  overview  of  the  field  and  the 
applications  of  MCDM. 

As  mentioned  previously,  participants  came  from  very  different  backgrounds.  This  led  to 
interesting  discussions  as  case  studies  were  presented  to  ‘theoreticians’.  In  a  few  instances, 
claims  made  during  the  presentations  were  refuted  by  one  or  more  members  of  the  audience, 
leading  to  instructive  discussions  for  both  the  presenter  and  the  audience. 

An  important  element  that  comes  out  of  this  conference  is  the  practical  impossibility  to 
measure  the  quality  of  a  decision.  This,  to  a  certain  extent,  limits  the  convergence  of  methods 
because  results  cannot  be  compared.  It  also  means  that  mistakes  or  wrong  assumptions  can 
easily  go  unnoticed,  enforcing  the  idea  that  the  best  decision  support  systems  must  be  sound 
and  simple. 
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List  of  symbols/abbreviations/acronyms/initialisms 


AHP 

Analytical  Hierarchy  Process 

CORT 

Central  Operational  Research  Team 

DES 

Discrete  Event  Simulation 

DRDC 

Defence  Research  and  Development  Canada 

IDS 

Intelligent  Decision  System 

MARVIN 

Maritime  Vignettes  (Discrete  Event  Maritime  Simulation) 

MAVT 

Multi-  Attribute  Value  Theory 

MCDM 

Multi-Criteria  Decision  Making 

MOLP 

Multi-Objective  Linear  Programming 

ORD 

Operational  Research  Division 
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